RESUMEN
PURPOSE: Nearly all published ophthalmology-related Big Data studies rely exclusively on International Classification of Diseases (ICD) billing codes to identify patients with particular ocular conditions. However, inaccurate or nonspecific codes may be used. We assessed whether natural language processing (NLP), as an alternative approach, could more accurately identify lens pathology. DESIGN: Database study comparing the accuracy of NLP versus ICD billing codes to properly identify lens pathology. METHODS: We developed an NLP algorithm capable of searching free-text lens exam data in the electronic health record (EHR) to identify the type(s) of cataract present, cataract density, presence of intraocular lenses, and other lens pathology. We applied our algorithm to 17.5 million lens exam records in the Sight Outcomes Research Collaborative (SOURCE) repository. We selected 4314 unique lens-exam entries and asked 11 clinicians to assess whether all pathology present in the entries had been correctly identified in the NLP algorithm output. The algorithm's sensitivity at accurately identifying lens pathology was compared with that of the ICD codes. RESULTS: The NLP algorithm correctly identified all lens pathology present in 4104 of the 4314 lens-exam entries (95.1%). For less common lens pathology, algorithm findings were corroborated by reviewing clinicians for 100% of mentions of pseudoexfoliation material and 99.7% for phimosis, subluxation, and synechia. Sensitivity at identifying lens pathology was better for NLP (0.98 [0.96-0.99] than for billing codes (0.49 [0.46-0.53]). CONCLUSIONS: Our NLP algorithm identifies and classifies lens abnormalities routinely documented by eye-care professionals with high accuracy. Such algorithms will help researchers to properly identify and classify ocular pathology, broadening the scope of feasible research using real-world data.
RESUMEN
PURPOSE: The purpose of this study was to develop a decision-support tool to predict anterior segment vision-threatening disease (asVTD) to aid primary care physicians (PCPs) with patient triage and referral. METHODS: The University of Michigan electronic health record data between January 1, 2016, and May 31, 2019, were obtained from patients presenting to a PCP with anterior eye symptoms and then saw an ophthalmologist within 30 days. asVTD included diagnosis of corneal ulcer, iridocyclitis, hyphema, anterior scleritis, or scleritis with corneal involvement by an ophthalmologist. Elastic net logistic regression with 10-fold cross-validation was used for prediction modeling of asVTD. Predictors evaluated included patient demographics and PCP notes processed using clinical natural language processing software (clinspacy). RESULTS: Two thousand nine hundred forty-two patients met the inclusion criteria, of which 133 patients (4.5%) had asVTD. The age was significantly lower among those with asVTD versus those without (median = 42 vs. 53 yrs, P < 0.001). Sex ( P = 0.8) and race ( P = 0.9) were not significantly different between groups. The final prediction model had an area under the curve of 0.72 (95% confidence interval 0.67-0.77). At a threshold achieving a sensitivity of 90%, the specificity was 30%, the positive predictive value was 5.8%, and the negative predictive value was 99%. CONCLUSIONS: The use of the prediction model increased the positive predictive value for asVTD compared with referral based on prevalence probabilities (17 patients vs. 22 patients needing to be evaluated to identify 1 case of asVTD). A prediction algorithm has potential to improve triage and initial management decision-making for PCPs because it performs better than probabilities in the absence of such a tool.
